Learning scalable multi-agent coordination by spatial differentiation for traffic signal control

The intelligent control of the traffic signal is critical to optimization transportation systems. To achieve global optimal efficiency in large-scale road networks, recent works have focused on coordination among intersections, which shown promising results. However, existing studies paid more attention observations sharing intersections (both explicit and implicit) did not care about consequences after decisions. In this paper, we design a multi-agent framework based Deep Reinforcement Learning method for control, defined as ?-Reward that includes both original ?-Attention-Reward. Specifically, propose Spatial Differentiation uses temporal–spatial information replay buffer amend reward each action. A concise theoretical analysis proves proposed model can converge Nash equilibrium given. By extending idea Markov Chain dimension space–time, truly decentralized mechanism replaces graph realizes decoupling network, scalable line with practice. simulation results show remains state-of-the-art performance even use centralized setting. Code available https://github.com/Skylark0924/Gamma_Reward.